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2.  Objectives.  The  primary  objective  is  to  advance  the  understanding  of  how  humans  use  vision  to  control  their 
locomotion  through  space.  The  research  deals  with  terrestrial  locomotion  (walking  and  driving)  and  aircraft 
piloting.  The  secondary  objective  is  to  contribute  to  improvements  in  driving  and  flight  safety.  The  objectives  are 
very  close  to  those  in  the  proposal  of  2001,  but  the  research  was  broadened  to  include  postural  control  and 
interception  tasks.  This  was  done  because  interesting  questions  about  these  came  up  in  the  course  of  the  research. 

3.  Status  of  effort.  The  accomplishments  were  threefold.  First,  a  software  tool  for  rendering  virtual 
environments  was  developed,  a  tool  useful  for  other  researchers  interested  in  visual  perception  and  visual 
control  of  action.  Second,  an  instrumented  electric  scooter  was  developed  that  allows  a  person  to  drive 
around  in  virtual  reality  while  experiencing  the  normal  inertial  cues  associated  with  physical  motion. 

This  type  of  research  vehicle  can  be  used  to  investigate  the  roles  of  visual  and  inertial  cues  in  the  control 
of  locomotion.  Third,  a  number  of  behavioral  experiments  were  conducted  in  real  and  virtual 
environments.  The  principal  research  findings  were  these:  (1)  complex  behaviors,  like  steering  a  curved 
path  and  ball  catching,  can  be  performed  without  the  retinal  motion  associated  with  luminance-based 
stimulation,  (2)  visual  control  of  posture  depends  on  the  sensed  relative  motion  between  self  and 
environment  instead  of  on  “optic  flow”,  (3)  people  can  continue  steering  a  vehicle  after  the  loss  of  visual 
information,  implicating  an  internal  model  of  surrounding  space,  (4)  a  steering  error  observed  while 
driving  with  visual  information  only  is  eliminated  when  inertial  cues  are  made  available,  and  (5)  a  flying 
study  reveals  the  challenge  airplane  pilots  face  when  pulling  up  from  a  dive  toward  terrain. 

4.  Accomplishments/New  Findings 

I.  Development  of  software  technique  for  presenting  cyclopean  stimuli. 

In  determining  which  visual  information  is  used  to  control  locomotion  and  other  forms  of  action,  a 
stimulus  which  contains  only  binocular  depth  information  (i.e.,  no  monocular  information)  is  of  value. 

This  type  of  stimulus,  referred  to  as  a  cyclopean  stimulus,  was  first  developed  by  Julesz  in  1971 .  It 
consists  of  a  rapid  sequence  of  random  dot  stereograms,  each  lasting  just  one  graphics  frame.  What 
appears  to  each  eye  as  just  a  scintillating  field  of  random  dots  of  uniform  spatial  density  appears  to  both 
eyes  as  a  world  of  stationary  and  moving  objects.  Using  the  primitive  computers  of  his  time,  Julesz 
(1971)  showed  that  a  subject  viewing  such  a  sequence  could  readily  perceive  smooth  motion  of  a  target  in 
the  frontoparallel  plane.  Gray  and  Regan  (1996,  1998)  reported  research  using  special  electronic 
hardware  used  in  conjunction  with  a  computer;  this  hardware  made  it  possible  to  move  a  single  target  in 
real  time  using  the  same  basic  technique  developed  by  Julesz.  They  showed  that  subjects  can  accurately 
judge  direction  of  motion  in  depth  as  well  as  time  to  collision  with  an  approaching  object.  Co-PI  Andrew 
Beall  has  developed  a  software  technique  that  can  be  used  with  any  modem  3-D  graphics  accelerator  card 
that  uses  OpenGL  graphics  functions.  This  means  that  any  computer-graphics  rendered  3-D  object, 
objects,  or  complex  scene,  whether  displayed  on  a  standard  desktop  computer  with  3-D  graphics  card  or 
with  a  head-mounted  display  in  immersive  virtual  can  be  presented  as  a  purely  binocular  (cyclopean) 
stimulus.  The  co-PI  made  his  software  plug-in  available  for  free  to  anyone  who  wishes  it,  by  way  of  his 
Web  site:  http://www.recveb.ucsb.edu/stickvdots.htm  The  software  is  simple  to  use;  with  a  single 
keypress  within  a  graphics  program,  a  person  can  switch  from  normally  rendered  3-D  graphics  to  a 
cyclopean  version. 

The  Gray  and  Regan  hardware  was  not  portable  and,  consequently,  few  people  have  been  able  to  use  this 
type  of  stimulus  in  their  research.  There  are  many  interesting  questions  concerning  visual  perception, 
visual  cognition,  and  visually  controlled  action  to  be  pursued  using  this  new  stimulus.  Thus,  this  new 
software  technique  is  a  valuable  resource  for  other  researchers. 
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II.  Development  of  an  instrumented  vehicle  for  use  in  an  immersive  virtual  environment 


A  new  research  tool  has  been  developed  in  accordance  with  the 
project  proposal  of  200 1 .  It  involves  an  electric  scooter  of  the 
kind  used  by  disabled  and  elderly  people  (see  figure).  The 
vehicle  can  move  at  up  to  5  mph  and  make  tight  turns  while 
being  virtually  impossible  to  tip  over.  This  means  that  the 
subject  is  kept  safe  while  being  exposed  to  suprathreshold 
inertial  cues  signaling  motion.  All  of  the  electronic  and 
computer  equipment  for  tracking  the  position  and  orientation  of 
the  vehicle  and  for  implementing  virtual  reality  is  carried  at  the 
rear  of  the  vehicle. 


The  subject  is  immersed  within  a  fully  interactive  virtual  environment  (wearing  a  head-mounted 
display)  while  moving  about  within  a  large  physical  space.  The  vehicle  has  been  instrumented  with  a 
potentiometer  attached  to  the  steering,  an  AD/DC  converter  to  sense  the  speed  control,  an  optical  encoder 
that  senses  wheel  turning,  and  a  high  quality  GPS  receiver  to  track  the  vehicle’s  location  out  of  doors  A 
software  Kalman  filter  integrates  the  various  inputs  to  give  an  accurate  and  precise  measurement  of  the 
vehicle’s  position  and  orientation  at  all  times,  even  when  GPS  signals  are  weak.  By  mounting  the  vehicle 
on  a  stationary  platform,  the  subject  can  visually  drive  the  vehicle  through  the  virtual  environment 
without  physically  moving.  By  comparing  performance  in  the  stationary  and  moving  conditions,  the 
vehicle  can  be  used  to  determine  the  contribution  of  inertial  cues  to  visual  control  of  vehicle,  both  with 
concurrent  visual  input  and  with  visual  input  temporarily  eliminated.  The  idea  of  using  a  moving  ground 
vehicle  as  the  platform  for  immersive  virtual  reality  is  unique  and  has  several  advantages  over  motion- 
base  simulators  for  studying  the  role  of  inertial  cues  in  vehicle  control.  Because  of  this,  other  researchers 
have  expressed  interest  in  the  vehicle  developed  for  this  project. 


III.  Visual  perception  and  visually  controlled  behaviors  without  retinal  optic  flow 

Much  research  over  the  years  has  shown  the  importance  of  optic  flow  in  the  visual  control  of  locomotion 
and,  more  generally,  action.  Optic  flow  (changes  in  angular  positions  of  environmental  points)  produced 
by  self-motion  gives  rise  to  corresponding  retinal  flow,  except  that  eye  rotations  cause  additional  retinal 
flow  that  is  superimposed  on  that  produced  by  optic  flow.  Using  the  software  technique  described  above, 
research  has  been  conducted  on  three  issues  relating  to  the  visual  control  of  action.  This  research  shows 
that  a  more  abstract  form  of  optic  flow  produced  by  cyclopean  stimulation  results  in  visual  perception  and 
visually  based  action  similar  to  those  resulting  from  the  more  conventional  luminance-based  optic  flow, 
which  has  retinal  motion  (“Fourier  motion”  or  “motion  energy”). 


A.  Visual  control  of  action  without  retinal  optic  flow.  In  three  publications  (Loomis  & 
Loomis  et  al.,  2005;  Loomis  et  al.,  in  press),  the  research  team  has  shown 
that  a  wide  variety  of  complex  actions  can  be  performed  using  cyclopean 
stimulation  (i.e.,  without  retinal  optic  flow)  about  as  well  as  using 
conventional  luminance  stimulation  (monocular  or  binocular).  For  these,  the 
subject  wears  a  head-mounted  display  and  moves  through  the  cyclopean 
version  of  a  virtual  environment.  These  actions  include  walking  to  a  goal 
through  a  field  of  obstacles,  ball  catching,  steering  a  curving  path,  and 
steering  a  straight  path  in  the  presence  of  lateral  perturbing  forces.  The 
figure  to  the  right  shows  a  subject  catching  a  ball  in  virtual  reality. 
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The  data  figure  on  the  right  shows  the  results  for 
ball  catching  with  different  visual  stimuli.  Ball 
catching  performance  (percent  caught)  was  almost 
as  good  with  the  purely  binocular  (cyclopean) 
stimulus  as  with  conventional  luminance  stimuli 
(both  binocular  and  monocular);  no  vision  and  a 
single  eye  view  of  the  cyclopean  stimulus  led  to 
the  same  poor  performance.  (For  the  latter  two 
conditions,  the  relatively  high  proportion  of 
successful  “catches”  occurred  because  the  landing 
region  of  the  ball  was  quite  small  and  a  successful 
catch  was  registered  when  the  subject  was  within 
some  critical  radius  of  the  ball). 

The  results  for  other  visually  guided  actions  were  similar  in  showing  that  cyclopean  stimulation  allows 
almost  the  same  level  of  performance  as  conventional  luminance  stimulation,  which  contains  informative 
retinal  motion.  This  work  shows  that  complex  actions  can  be  carried  out  visually  using  a  more  abstract 
form  of  optic  flow  that  does  not  involve  retinal  motion.  However,  because  some  actions,  like  playing 
tennis  and  batting  a  baseball,  involve  optic  flow  of  high  angular  velocities,  cyclopean  visual  perception  is 
probably  inadequate  for  these  given  that  its  spatiotemporal  bandwidth  is  lower  than  that  of  luminance- 
based  vision. 


B.  Heading  perception  without  retinal  optic  flow.  Many  psychophysical  studies  in  the  1990's 
investigated  “heading  perception”  with  optic  flow  presented  on  computer  displays.  Project  personnel 
have  conducted  research  on  heading  perception  without  retinal 
optic  flow.  Using  the  traditional  paradigm  for  measuring  the 
precision  of  heading  perception  (except  that  the  subject  wears 
a  head-mounted  display)  and  adjusting  the  luminance  stimuli 
to  be  equal  to  the  cyclopean  stimuli  in  terms  of  motion 
visibility,  it  was  found  that  heading  thresholds  for  cyclopean 
stimuli  were  nearly  the  same  as  those  for  luminance  (“optic 
flow”)  stimuli;  in  the  accompanying  figure,  the  thresholds 
(corresponding  to  75%  percent  correct  performance)  were  just 
slightly  larger  for  the  two  observer  speeds  studied  (Macuga  et 
al.,  submitted). 


C.  Postural  control  without  retinal  optic  flow.  Prior  research  has  demonstrated  that  postural  sway  can 
be  induced  by  optic  flow,  as  for  example,  when  a  swinging  room  surrounds  a  person  (Lee  &  Lishman, 
1975).  Project  personnel  conducted  research  to  determine  whether  such  optic  flow  needs  to  involve 
retinal  flow  or  can  be  more  abstract,  like  that  produced  by  cyclopean  stimulation.  In  this  research,  the 
subject  stood  upright  wearing  a  head-mounted  display.  The 
goal  was  to  move  as  little  as  possible,  while  a  large  cross 
moved  sinusoidally  toward  and  away  from  the  subject.  Its 
motion  was  conveyed  by  cyclopean  or  luminance  stimuli;  in 
the  latter  case,  the  stimulus  was  biocular  (identical  for  the 
two  eyes).  When  adjusting  the  luminance  stimuli  to  match 
the  cyclopean  stimuli  in  terms  of  motion  visibility,  similar 
(but  small)  amounts  of  body  sway  were  observed  (see  figure 
at  the  right).  Both  were  greater  than  the  amount  of  sway 
measured  when  the  subject  was  asked  to  stand  still  while 
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viewing  a  stationary  stimulus  defined  by  luminance.  This  work  shows  that  the  optic  flow  inducing 
postural  sway  need  not  have  associated  retinal  flow. 

IV.  Research  showing  that  optic  flow,  even  in  the  abstract  sense,  is  not  the  basis  for  postural 
control. 

While  much  research  has  been  reported  claiming  that  optic  flow  is  the  visual  basis  for  postural  control, 
there  is  an  alternative  hypothesis.  It  is  that  the  subject  perceives  relative  movement  between  self  and  the 
environment  (while  standing  still)  and  makes  appropriate  postural  adjustments.  Usually,  when  the  self  is 
moving  relative  to  the  environment,  there  is  corresponding  optic  flow,  either  in  the  abstract  sense 
mentioned  above  or  in  the  conventional  sense,  which  involves  retinal  flow  as  well.  To  distinguish  these, 
one  can  manipulate  binocular  cues,  which  can  alter  the  perceived  motion  between  self  and  environment 
without  altering  the  optic  flow.  In  the  research  on  this  topic  (Kelly,  Loomis,  &  Beall,  2005),  the  subject 
was  made  unstable  by  having  to  standing  on  one  foot.  In  one  visual  condition  the  visual  stimulus  was  a 
rectangular  box  viewed  binocularly  in  immersive  virtual  reality.  This  box  appears  to  be  rigid  and 
stationary  as  the  person’s  head  translates  slightly  (because  of  postural  instability).  The  other  condition 
was  obtained  simply  by  switching  the  images  to  the  two  eyes.  Switching  the  eyes  has  no  effect  on  the 
optic  flow  but  dramatically  changes  the  perception  of  the  rectangular  box  in  terms  of  3-D  perception. 
Now  the  box  appears  as  a  truncated  pyramid, 
which  appears  to  rotate  and  distort  as  the 
person’s  head  moves  in  space.  The  apparent 
motion  and  non-rigidity  of  an  object  associated 
with  head  movement  has  been  called  “apparent 
concomitant  motion”  by  Gogel,  who  has  studied 
it  extensively  and  provided  a  detailed 
explanation  (Gogel,  1990).  In  two  experiments 
on  this,  it  was  found  that  such  a  stimulus  was 
much  less  effective  in  stabilizing  the  subjects 
(mean  and  individual  data  are  shown  in  the 
accompanying  figure;  for  the  two  experiments, 
amount  of  head  instability  is  plotted  as  a 
function  of  visual  condition  and  axis  of 
movement,  either  lateral  or  anterior/posterior). 

Because  the  optic  flow  was  unchanged,  this 
result  indicates  that  optic  flow  is  not  the  stimulus  that  controls  posture.  Rather,  the  postural  control 
system  is  relying  on  the  apparent  relative  motion  between  self  and  environment.  This  finding  is  an 
important  challenge  to  researchers  who  attempt  to  interpret  visual  control  of  action  solely  in  terms  of 
optic  flow  (whether  abstract  or  involving  retinal  flow). 

V.  Visual  control  of  steering. 

A.  Open-loop  steering.  In  line  with  the  2001  proposal  and  with  ideas  presented  in  the  chapter  by 
Loomis  and  Beall  (2004),  research  has  been  conducted  on  the  roles  of  both  optic  flow  and  a  perceptual 
model  of  the  path  being  steered  (Macuga,  Beall  et  al,  in  preparation).  The  experiment  involved  steering  a 
curved  path  defined  by  smooth  lane  markers.  A  driving  simulator  with  wide  projection  screen  was  used. 
An  important  manipulation  was  intermittent  presentation  of  the  path,  ranging  from  continuously  present 
to  one  video  frame  every  couple  of  seconds.  In  the  accompanying  figure  on  the  next  page,  the  different 
conditions  of  intermittency  over  time  are  presented  using  black  and  white  symbols,  with  the  white 
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representing  the  period  of  visibility.  Each  symbol  represents 
2.5  sec.  There  were  three  conditions  of  optic  flow  in  the 
experiment:  In  the  CONSISTENT  FLOW  condition,  the 
optic  flow  produced  by  ground  texture  was  consistent  with 
the  path  being  steered.  In  the  INCONSISTENT  FLOW 
condition,  the  optic  flow  had  two  components.  One 
component  corresponded  to  motion  over  a  separate  “ghost” 
path.  The  other  component  was  the  rotational  flow 
associated  with  the  subject’s  actual  steering  inputs.  Because 
the  changes  in  optic  flow  were  partially  correlated  with  the 
steering  inputs,  the  subject  could  use  the  changes  in  flow  to 
gauge  turn  rate.  The  third  and  final  flow  condition  was  NO 
FLOW.  In  this  condition,  flow  was  consistent  but  it 
disappeared  when  the  path  disappeared. 


Steering  error 


The  results  of  the  three  conditions  were  quite  revealing.  For  continuous  viewing  of  the  path,  there  was 
little  effect  of  whether  flow  is  consistent  or  inconsistent.  For  intermittent  viewing,  CONSISTENT  FLOW 
led  to  best  steering  performance,  as  expected.  What  was  surprising  is  that  the  NO  FLOW  condition  was 
much  worse  than  INCONSISTENT  FLOW.  This  means  that  although  the  INCONSISTENT  FLOW  does 
not  specify  the  vehicle’s  absolute  heading,  the  changing  flow  (which  is  partly  determined  by  the  subject’s 
steering  inputs)  specifies  the  turn  rate  of  the  vehicle,  which  can  be  used  by  the  subject  for  steering.  These 
results  are  interpreted  in  the  following  way.  Subjects  are  using  path  information  that  is  continuously  or 
intermittently  present  to  form  a  3-D  or  perspective  representation  of  the  path.  Subjects  steer  with  respect 
to  the  perceptual  path  when  possible,  but  are  able  to  mentally  update  with  respect  to  the  imagined  path 
when  it  is  not  present.  The  optic  flow  serves  to  provide  information  about  the  speed  and  turns  of  the 
vehicle  even  when  absolute  information  about  heading  is  provided  by  optic  flow. 


If  subjects  are  indeed  updating  with  respect  to  the  imagined  path  when  it  is  not  perceptually  present,  they 
should  make  several  successive  steering  inputs  in  accord  with  the  changing  heading  of  the  path,  even 
when  they  cannot  see  the  path.  In  recent  work,  this  has  been  confirmed  (Macuga,  Loomis  et  al.,  in 
preparation).  The  accompanying  figure  depicts  a  planview  of  a  curved 
path  (solid  line)t.  The  dashed  line  is  the  average  of  8  steering  trajectories 
(4  subjects  twice  each)  performed  without  vision.  The  subjects  moved 
along  a  straight  segment  while  viewing  the  path  ahead  and  just  as  they 
entered  the  curved  path,  the  path  disappeared,  leaving  only  optic  flow  of 
ground  texture.  The  beginning  of  the  average  steering  trajectory  is  at  the 
bottom.  This  figure  is  just  one  of  7  different  paths.  This  and  figures  for 
the  other  paths  show  that  although  subjects  are  not  steering  with  perfect 
accuracy,  they  are  making  multiple  steering  inputs  at  appropriate  times  to 
maintain  an  average  path  that  is  quite  similar  to  the  actual  path.  This  is 
evidence  of  imaginal  updating  with  respect  to  a  mental  representation  of 
the  path. 


These  new  results  also  contrast  with  those  of  several  experiments  by  Wallis  et  al.  (2002),  in  which  they 
found  that  subjects  making  a  lane  change  on  a  driving  simulator,  make  a  steering  input  to  move  toward 
the  next  lane  but  fail  to  make  the  opposite  steering  input  to  realign  with  the  path.  From  this  result  they 
have  concluded  that  people  are  unable  to  perform  open  loop  steering,  in  part  because  of  their  having  a 
poor  representation  of  the  steering  dynamics  of  a  vehicle.  The  results  above  challenge  the  generality  of 
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their  result  and  their  interpretation  of  it.  However,  because  the  paths  in  the 
above  study  were  curved,  it  was  necessary  to  investigate  open  loop  steering 
in  a  task  more  like  theirs.  In  recent  research,  subjects  had  to  perform 
several  steering  tasks,  of  which  just  two  are  mentioned  here.  One  was  a 
lane  change  maneuver  like  that  of  Wallis  et  al.  (2002).  Subjects  were 
moving  down  a  two  lane  road  and,  on  command,  were  instructed  to  change 
over  into  the  adjacent  lane.  At  the  moment  of  the  command,  the  path 
disappeared.  In  the  other  condition,  subjects  viewed  a  path  consisting  of  3 
linear  segments  (corresponding  approximately  to  a  lane  change  maneuver). 
As  the  approached  the  path,  it  disappeared.  The  accompanying  figure 
gives  results  of  multiple  attempts  by  one  subject  for  the  two  conditions 
(lane  change  is  above).  The  results  of  9  subjects  are  similar  in  showing 
that  when  subjects  make  a  lane  change,  they  show  the  absence  of  a  ‘'return 
phase”  in  accord  with  the  results  of  Wallis  et  al.  (2002)  but  when  there  is 
an  explicit  path  depicted,  subjects  are  able  to  perform  open  loop  steering 
that  is  qualitatively  correct. 


In  the  most  recent  research  on  this  topic,  subjects  performed  steering  tasks 
on  the  instrumented  electric  scooter  mentioned  earlier  out  of  doors. 

Subjects  wore  a  head-mounted  display  and  saw  paths  depicted  in  a  virtual  environment.  While  at  a 
standing  start,  they  saw  either  a  two  lane  straight  path  ahead  or  an  explicit  path  of  3  linear  segments. 
When  ready,  the  subjects  accelerated  and  attempted  to 
make  a  lane  change  or  steer  the  path  they  had  seen. 

On  some  trials  the  display  was  turned  off  while  they 
were  moving  (open  loop).  The  results  are  shown  at  the 
right.  The  solid  lines  are  closed-loop  control  (with 
concurrent  visual  information).  Subjects  correctly 
performed  the  return  phase  for  both  steering  tasks  and 
for  both  open-  and  closed-loop  control.  The  results 
show  that  the  inertial  cues  provided  while  moving  in  a 
real  vehicle  are  sufficient  to  induce  the  subjects  to 
make  the  return  phase,  thus  eliminating  the  steering 
error  observed  by  Wallis  et  al. 


So 
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B.  Simultaneous  measurement  of  heading  perception  and  steering  performance 

Notwithstanding  the  extensive  literature  from  the  1990’s  on  heading  perception,  a  number  of  researchers 
have  expressed  skepticism  that  people  use  perceived  heading  based  on  optic  flow  to  perform  various 
steering  behaviors,  including  aiming  toward  a  point,  steering  a  straight  path  in  the  presence  of  laterally 
perturbing  forces,  and  turning  into  alignment  with  a  straight  path  (Beall  &  Loomis,  1996,  1997;  Loomis 
&  Beall,  1998;  Wann  &  Wilkie,  2004;  Rushton  &  Harris,  2004).  To  address  the  issue  of  whether 
perceived  heading  is  involved  in  steering  a  curved  path,  an  experiment  was  conducted  in  which  subjects 
steered  along  curved  paths  within  an  immersive  virtual  environment  (implemented  with  a  head-mounted 
display)  while  periodically  being  asked  to  make  judgments  of  instantaneous  heading  using  the  standard 
probe  technique.  The  paths  consisted  of  concatenated  segments  that  were  straight,  of  low  curvature,  or  of 
high  curvature.  Also  manipulated  was  the  magnitude  of  crosswinds.  Steering  performance  was  measured 


Closed-loop 

Open-loop 
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for  the  different  path  segments  in  terms  of  bias  (constant  lateral 
error)  and  variability  (standard  deviation  of  lateral  error).  Errors 
in  perceived  heading  were  measured  in  the  conventional  way. 

The  results  for  the  8  subjects  are  shown  in  the  accompanying 
figure.  Heading  bias  (which  could  only  be  measured  by 
collapsing  across  crosswind  magnitude)  varied  systematically 
with  the  direction  and  magnitude  of  the  path  curvature.  Steering 
bias  was  affected  in  a  very  similar  fashion.  The  high  correlation 
between  heading  bias  and  steering  bias  is  quite  surprising  and 
seems  to  support  the  hypothesis  that  perceived  heading  is 
involved  in  steering  a  curved  path.  However,  there  are 
alternative  hypotheses  that  might  account  for  such  a  correlation. 
Ultimately,  what  is  needed  is  a  theory  connecting  the  two. 

VI.  Sensitivity  to  visual  information  during  pullup  from  a 
vertical  dive 

One  of  the  most  complex  visually  controlled  maneuvers  is  that  of 
pulling  up  from  a  dive  in  an  airplane.  Failure  to  perform  this 
correctly  is  the  cause  of  occasional  accidents  at  airshows, 
including  one  in  2003  by  the  solo  pilot  of  the  USAF 
Thunderbirds  (http://www.f-l  6.net/f-l  6_news_article968.html). 
The  precipitating  cause  of  the  accident  was  the  pilot’s  briefing 
for  the  wrong  field  elevation,  but  his  failure  to  quickly  realize  his 
error  during  the  pullup  shows  how  difficult  the  maneuver  is;  he 
safely  ejected  less  than  1  sec  before  the  aircraft  crashed. 


!• 


U 


il 


There  is  no  instantaneous  visual  information  (in  the  form  of  optic 
flow  or  distance  cues)  that  informs  the  pilot  as  to  whether 

collision  with  the  ground  will  occur.  Loomis  and  Beall  (1998)  speculated  that  one  basis  forjudging 
involved  changes  over  time  of  the  aim  point,  as  specified  by  optic  flow.  In  particular,  it  was  hypothesized 
that  the  aircraft  would  pass  clear  if  and  only  if  the  aim  point  accelerated  positively  over  the  ground 
surface.  Project  member,  Professor  Roy  Smith,  did  some  mathematical  modeling  and  found  that  the 
hypothesis  is  not  quite  correct.  The  accompanying 
figure  shows  the  acceleration  of  the  aim  point  as  a 
function  of  the  angle  in  a  circular  dive  trajectory 
(when  the  aircraft  is  diving  vertically  at  the  start, 
the  angle  is  -90°).  Trajectories  that  collide  always 
have  negative  accelerations.  Trajectories  that  will 
pass  clear  initially  have  negative  accelerations,  but 
eventually  the  aim  point  switches  to  a  positive 
acceleration.  The  diagnostic  information  occurs 
late  in  the  trajectory,  which  may  be  only  a  few 
seconds,  depending  upon  the  rate  of  pullup. 


Starting  in  February  2004,  a  control  theory  student  from  Delft  University,  Sebastiaan  de  Stigter,  served  as 
a  research  intern  in  the  lab  for  3  months.  Besides  giving  tutorials  on  control  theory,  he  conducted 
experimental  research  on  the  pullup  maneuver.  Although  the  data  have  yet  to  be  thoroughly  analyzed,  the 
results  confirm  the  difficulty  of  judging  whether  the  trajectory  will  pass  clear  and  reveal  an  important  bias 
associated  with  the  pitch  of  the  aircraft,  suggesting  that  people  have  difficulty  distinguishing  between  the 
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pointing  direction  of  the  aircraft  and  its  instantaneous  travel  direction.  This  result  may  have  implications 
for  pilot  training. 


5.  Personnel  supported  by  or  associated  with  project 


Jack  Loomis 
Andrew  Beall 
Roy  Smith 
Jerome  Tietz 
Jonathan  Kelly 
Kristen  Macuga 
Sebastiaan  de  Stigter 
Thomas  Cook 


Professor  of  Psychology;  Principal  Investigator 
Project  scientist;  co-Principal  Investigator 

Professor  of  Electrical  and  Computer  Engineering;  Senior  Personnel 
Computer  programmer 
Graduate  student  researcher 
Graduate  student  researcher 

Undergraduate  intern  from  Delft  University,  The  Netherlands 
Undergraduate  engineering  student  from  UCSB 


6.  Publications  and  manuscripts  deriving  from  AFOSR  support  during  the  grant  period 

Loomis,  J.  M.  &  Beall,  A.  C.  (2004).  Model-based  control  of  perception/action.  In  L.  Vaina,  S. 
Beardsley,  and  S.  Rushton  (Eds.).  Optic  Flow  and  Beyond  (pp.  42 1  -44 1 ).  Boston:  Kluwer  Academic 
Publishers. 


Kelly,  J.  W.,  Loomis,  J.  M.,  &  Beall,  A.  C.  (2004).  Judgments  of  exocentric  direction  in  large-scale 
space.  Perception ,  33,  443-454.  (also  supported  by  Office  of  Naval  Research) 

Kelly,  J.W.,  Loomis,  J.M.,  &  Beall,  A.C.  (2005).  The  importance  of  perceived  relative  motion  in  the 
control  of  posture.  Experimental  Brain  Research ,  161,  285-292. 

Loomis,  J.  M.,  Beall,  A.  C.,  Kelly,  J.  W.,  &  Macuga,  K.  L.  (2005).  Importance  of  perceptual 
representation  in  the  visual  control  of  action.  Proceedings  of  the  IS&T/SPIE ’s  1 7lh  Annual  Symposium  on 
Electronics  (  pp.  356-361),  January  16-20,  2005,  San  Jose  CA. 

Loomis,  J.  M.,  Beall,  A.  C.,  Macuga,  K.  L.,  Kelly,  J.  W.  &  Smith,  R.  (in  press).  Visual  control  of  action 
without  retinal  optic  flow.  Psychological  Science. 

Macuga,  K.  L.,  Loomis,  J.  M.,  Beall,  A.  C.,  &  Kelly,  J.  W.  (under  revision;  acceptance  expected). 
Perception  of  heading  without  optic  flow.  Perception  &  Psychophysics. 

Kelly,  J.  W.,  Beall,  A.  C.,  Loomis,  J.  M.,  Smith,  R.  S.,  &  Macuga,  K.  L.  (under  revision;  acceptance 
expected).  Simultaneous  measurement  of  steering  performance  and  perceived  heading  on  a  curving  path. 
ACM  Transactions  on  Applied  Perception. 

Macuga,  K.  L.,  Beall,  A.  C.,  Loomis,  J.  M.  &  Smith,  R.  S.  (in  preparation).  Visual  control  of  steering 
along  a  curving  path. 

Macuga,  K.  L.,  Loomis,  J.  M.,  Beall,  A.  C.,  Smith,  R.  S.,  &  Kelly,  J.W.  (in  preparation).  In  steering 
without  visual  feedback,  subjects  can  properly  initiate  the  return  phase  of  a  “lane  change”  maneuver. 

de  Stigter,  S.,  Loomis,  J.  M.,  Smith,  R.,  &  Beall,  A.  C.  (in  preparation).  Psychophysical  investigation  of 
sensitivity  to  visual  information  during  pullup  from  a  vertical  dive. 
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Kelly,  J.  W.,  Beall,  A.  C.,  Loomis,  J.  M.,  &  Macuga,  K.  L.  (in  preparation).  Postural  control  without 
retinal  optic  flow. 

7.  Interactions/Transitions 

a.  Talks  and  posters  relating  to  AFOSR  funding  (in  chronological  order) 

Beall.  A.  Creating  virtual  environments.  Thematic  Interdisciplinary  School  on  Virtual  Reality  and 
Behavioral  Science.  Marseille  France,  May  2003. 

Loomis,  J.  Using  virtual  reality  to  study  visual  space  perception,  action  and  spatial  cognition.  Thematic 
Interdisciplinary  School  on  Virtual  Reality  and  Behavioral  Science.  Marseille  France,  May  2003. 

Kelly,  J.,  Beall,  A.,  &  Loomis,  J.  (2003).  Postural  control  without  optic  flow.  Vision  Sciences  Society 
meeting,  Sarasota,  FL,  2003. 

Loomis,  J  &  Beall,  A.  (2003).  Visual  control  of  locomotion  without  optic  flow.  Vision  Sciences  Society 
meeting,  Sarasota,  FL,  May  2003. 

Macuga,  K.,  Loomis,  J.  &  Beall,  A.  (2003).  Perception  of  heading  without  optic  flow.  Vision  Sciences 
Society  meeting,  Sarasota,  FL.,  2003 

Loomis,  J.  Visual  control  of  locomotion.  Colloquium,  Schepens  Eye  Research  Institute,  Boston,  MA, 
July,  2003. 

Loomis,  J.  Perceiving  and  acting  in  space.  Short  course.  International  Spatial  Cognition  Institute,  Bad 
Zwischenahn,  Germany,  September  2003. 

Loomis,  J.  Perceiving  and  acting  in  real  and  virtual  environments.  Colloquium,  Department  of 
Neurosciences,  School  of  Medicine,  Erasmus  University,  Rotterdam,  The  Netherlands,  September  2003. 

Loomis,  J.  Visual  control  of  locomotion.  Colloquium,  Faculty  of  Aerospace  Engineering,  Delft  University 
of  Technology,  Delft,  The  Netherlands,  September  2003. 

Kelly,  J.  Postural  control  without  optic  flow.  Colloquium,  Max-Planck  Institute,  Tubingen,  Germany, 
September  2003 

Loomis,  J.  Perceiving  and  acting  in  space.  Colloquium,  Dept,  of  Psychological  and  Brain  Sciences, 
Johns  Hopkins  University,  Baltimore,  MD,  October  2003. 

Loomis,  J.  M.  (2004).  Visual  control  of  flight.  Aerospace  Lighting  Institute,  Los  Angeles,  CA,  February 
2004. 

Kelly,  J.W.  (2004).  Perception  and  action  without  optic  flow.  Colloquium,  University  of  Texas  at 
Austin. 

Kelly,  J.W.,  Beall,  A.C.,  &  Loomis,  J.M.  (2004).  Accurate  steering  performance  with  large  heading 
errors  on  a  curving  path.  Vision  Sciences  Society  Annual  Meeting,  Sarasota,  FL. 
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Macuga,  K.  L.,  Loomis,  J.  M.,  &  Beall,  A.  C.  (2004).  Two  processes  in  the  visual 

control  of  steering  along  a  curving  path:  sensing  turns  and  updating  with  respect  to  the  path.  Vision 

Science  Society  Annual  Meeting,  Sarasota,  FL.,  2004. 

Loomis,  J.  M.  Recent  research  on  spatial  hearing  inspired  by  working  with  blind  people.  Colloquium, 
Auditory  Research  Group,  Wright-Patterson  Air  Force  Base,  Dayton,  OH,  October  2004. 

Loomis,  J.  Perceiving  and  acting  in  space.  Colloquium,  Dept,  of  Psychology,  Wright  State  University, 
Dayton,  OH,  October  2004. 

Loomis,  J.  Perceiving  and  acting  in  space.  Colloquium,  Dept,  of  Psychology,  Rutgers  University, 
Newark,  NJ,  December  2004. 

Macuga,  K.  L.,  Loomis,  J.  M.,  Beall,  A.  C.,  Smith,  R.,  &  Kelly,  J.W.  (2005).  In  steering  without  visual 
feedback,  subjects  can  properly  execute  return  phase  of  a  “lane  change”  maneuver.  Journal  of 
Vision.  Presented  at  Vision  Science  Society  Annual  Meeting,  Sarasota,  FL,  May  6-11,  2005. 

b.  Consultative  and  advisory  functions  to  Air  Force  laboratories 

In  October,  2004,  the  PI,  Jack  Loomis,  traveled  to  Wright-Patterson  Air  Force  Base  and  spent  the  day 
with  the  Auditory  Research  Group,  where  he  gave  a  talk  and  discussing  with  various  personnel  there 
possible  new  directions  for  research  in  the  area  of  spatial  hearing. 

c.  Transitions  None 

8.  New  inventions 

Instrumented  electric  scooter  that  permits  a  user  to  be  perceptually  immersed  in  a  virtual  environment 
while  driving  around  in  real  space. 

9.  Awards  None 
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